METHOD AND SYSTEM FOR INFORMATION ASSOCIATION 



AREA OF INVENTION 

The present invention relates to a method and system for 
information association and more particular to association of 
information written by hand and converted to digital format by 
a handheld device . 
BACKGROUND ART 

Information has been noted for hand by pen on paper since 
several hundreds of years. However, paperbased information has 
drawbacks, such as being permanent and difficult to change and 
distribute . 

Recently, in personal computers, information may be ente- 
red by keybords and displayed by display devices, like CRT or 
LCD screens. However, computerbased information has drawbacks, 
such as being difficult to enter by keyboards and difficult to 
read on a computer screen. 

Several suggestions exists for digitizing handwritten 
information for transferring such handwritten information to 
digital devices, such as computers. 

As soon as the information has been entered in the 
computer, it may be communicated to others, such as by e-mail 
or SMS or via an Internet connection or as a fax via a fax 
modem. 

In a computer context, a mouse is often used for selecting 
information and acting upon such selected information. A typi- 
cal situation involves draging a mouse pointer over characters 
of information displayed on the screen to select said informa- 
tion. Then, the mouse pointer is pointed at a certain position 
on the screen, such as in a menue system or pallette, in order 
to initiate certain actions to be performed upon the selected 
text, such as changing the font type or size. 

The same type of selection may be performed in the pen and 
paper situation, such as encircling a piece of information or 
yellow-mark information. However, since the information is 
fixed, no action can normally be taken upon the selected in- 
formation, which however, may be highlighted to the reader. 

Patent Application No. PCT/SEO 0/ 0 18 95 , filed by applicant, 



the technical disclosure of which is incorporated herein by 
reference, discloses a system in which a handheld device and a 
paper is used for jotting down information in a traditional 
way, whereby at the same time is generated a digital graph 
5 comprising one or several traces of lines of the movement of 
the pen over the paper, which graph may be transferred to a 
computer. In such a system, the advantages of using a pen and 
paper, which many users prefer, is combined with the flexi- 
bility of the computer to communicate and store the informa- 

10 tion to obtain the best of the two worlds. In this patent 

application, the paper is provided with an absolute position 
coding pattern, comprising dots or markings. The handheld 
device includes an optical sensor which registers the pattern. 
A mathematical algorithm calculates the absolute position in 

15 the form of coordinates for the position of the handheld 
device by means of the absolute positon coding pattern. 

The traditional pen is in this way tranformed into an 
input means for the computer and the computer may be used for 
storing the information instead of or in combination with 

2 0 arranging the paper in a binder. Moreover, the information may 

be communicated by means of the computer. However, the infor- 
mation inserted by this input method is graphical and not in 
character coded format, which the computer is more apt to. 

The information included in the graphical input comprises 
25 information which may be used for different purposes: 

1) The information comprises a picture, like drawings or 
lines with a mutual relationship, which may be interpreted by 
the human being, such as characters, symbols, or a drawing. 
This is the very information which is put down on the paper 

3 0 and which the user wants to be treated in one or the other 

way, such as being archived or being sent to a recepient . This 
information is below called message information and is stored 
in a graphical format such as a vector format or as a collec- 
tion of pixels. 

35 2) The portion of the message information which comprises 

handwritten characters, can be exposed to further treatment, 
such as character recognition (OCR, optical character recogni- 
tion or ICR, intelligent character recognition) for converting 



the information to a character format, such as ASCII, which 
may be used by the computer, e.g. for searching purposes or 
for cataloging. Also symbols, such as shorthand symbols or 
icons, may be recognized which by the user has been predefined 
5 to have a certain meaning. This information is below called 
character information. 

3) The information may further comprise an indent if icat ion 
of the handheld device which is used for writing the informa- 
tion. The identification of the handheld device is below 

10 called pen-ID 

4) Furthermore, there is information on where at the 
surface the information was jotted down, so called absolute 
position information. 

5) There may be further information, such as time informa- 
15 tion obtained from an internal clock, pressure information 

from a pressure sensor about the pressure of the handheld 
device towards the paper, and angle information about the 
angle between the handheld device and the paper. From the time 
information, speed and acceleration of the handheld device may 
2 0 be calculated. 

The present invention is mainly focused on the information 
of type 4) and 5) above, such as absolute position informa- 
tion . 

There are further known systems for obtaining absolute 

2 5 position information in a pen and paper system, see for 

example US 5,882,434 and US 5,051,736. Such previously known 
systems describe use of such information for digitizing messa- 
ge information, i.e. group 1) indicated above. 

The information may be used in different places in the 

3 0 system so far described. The handheld device may comprise a 

picture sensor and a processor with associated memory and an 
energy source such as a battery. Moreover, there may be a 
communication device, such as a wire or a connection to a 
computer or an IR link or a shortrange radio connection. There 
35 is also a pressure sensor which senses if the handheld device 
is applied to the paper and the pressure between the handheld 
device and paper. 

It is possible to transfer the information from the hand- 



held device to an external computer in the nature of a video 
signal from the sensor and all computing taking place in the 
external computer. Alternatively, the handheld device may be 
provided with means for image processing. Thus, the handheld 
5 device may be provided with a processor and/or logic arrays, 
which processes the video signal obtained from the sensor and 
calculates the absolute position from the dots or markings 
imaged from the surface of the paper. This image processing 
comprises adjustments for perspective effects from rotation 

10 and inclination of the handheld device in relation to the 
paper and compensation for different light conditions. The 
handheld device may further comprise computer programs which 
calculates the x-y-coordinates of the handheld device on an 
imaginary surface . 

15 Information may be stored in the memory of the handheld 

device in the nature of a coordinate train, which may be 
converted to a vector format. This message information may be 
transferred to a computer, which draws a graph on the screen 
of the computer by means of a computer program. The message 

2 0 information may be sent to a local or distant printer for 

direct printout or to a dumb display device which only has the 
ability to draw the graph at the screen, such as a TV screen. 

Interpretation of the message information to a character 
format may take place in the handheld device or in a computer 

2 5 system. 

In addition to coordinate information there is also time 
information, such as time information for each coordinate, 
which is recorded each hundredth of a second when the pressure 
sensor is activated. Moreover, there may be information about 
30 the pressure towards the support and inclination and rotation 
information for the position of the handheld device in rela- 
tion to the support . 

Message information may be communicated to a recepient in 
the nature of a fax or an e-mail. In this situation, the hand- 

3 5 held device may cooperate with a mobile telephone via a wire, 

IR or radio link. The mobile telephone may operate as a 
"modem" for transferring the message information to a telefax 
apparatus at a telefax number, or to Internet via calling a 



modem pool of an Internet operator. In this case, it may be 
required that certain portions of the message information is 
character recognized, which may take place in the handheld 
device. Such recognized information may be used as a telefax 
5 number or an IP address for communication via Internet. 

Address information in the mobile telephone may alternatively 
be used. 

The computer may perform the same functions as described 
above as performed in the handheld device . 

10 The information may be transferred to a server via 

Internet at a preprogrammed IP address and the character 
recognition may take place in said server. 

In the above situations, it may be desired to use the 
input device, i.e. the handheld device for selecting certain 

15 text and indicate some actions to be performed on such infor- 
mation. Actions may be qualifying actions, such as qualifiying 
certain properties of the selected information, such as color 
or line thickness etc. Actions may also be operations to be 
performed on the selected information, such as character 

20 recognition or encryption. Other types of actions may be to 
expose the information for a program for encapsulating the 
information to a e-mail standard format to be transmitted as 
an e-mail. Further types of actions may be to compile the 
selected information in a binder or folder in a computer file 

25 system, as further described below. 

Thus, there is a need for a method and a system for 
associating information with other information, actions or 
qualifications . 

DISCLOSURE OF THE INVENTION 

30 A first object of the invention is to use absolute 

position coordinates for providing a method and a system for 
associating information as described above to make such 
association simple and intuitive. 

A second object of the invention is to provide a method 

3 5 and system for digitizing handwritten information by means of 
a handheld device and associating other information, actions 
or qualifications to said information by using said handheld 
device . 
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A third object of the invention is to provide a method and 
a system as described above, which takes advantages of an 
absolute postion coding pattern, which codes coordinates for a 
postion of the handheld device. 
5 A fourth object of the invention is to provide a method 

and system for providing a partly discontinuous absolute posi- 
tion coding pattern, whereby the discontinuity is used for 
said association. 

According to an aspect of the invention, there is provided 

10 a method and system for generating an association with regard 
to information, which is arranged on at least one support 
provided with an absolute position coding pattern and by means 
of a handheld device arranged to sense said absolute position 
coding pattern. The handheld device is passed over said 

15 support for sensing said absolute position coding pattern. 
When it is sensed that the handheld device passes over a 
discontinuity in the absolute position coding pattern such 
discontinuity is interpreted as an association regarding the 
information present on said support. 

2 0 The support may comprise a first area in which said 

absolute position coding pattern codes continuous first 
coordinates. The information is provided on the first area and 
is connected to the first coordinates. Moreover, the support 
comprises a second area provided with an absolute position 
25 coding pattern which codes second coordinates which are 

discontinuous with the first coordinates. When the handheld 
device is passed in a single movement from one area to the 
other area, the discontinuity is sensed. 

The association may involve that a predetermined action is 

3 0 undertaken on the information, such as sending the information 

as a message, such as an e-mail, SMS or fax, to a recipient. 
The recipient may be defined in the information or be defined 
by the association. In the last case, the recipient may be 
defined by the coordinates in said second area, which coordi- 
3 5 nates are connected to information about the recipient. 

The association may involve that the information is stored 
at a predetermined location in a computer system. Such loca- 
tion may be a predetermined folder in a personal computer 



defined by the coordinates of the second area. 

The association may involve that an operation may be 
performed on the information, such as encryption or character 
recognition. 

5 The association may involve that a qualification is 

imposed on the information. Such qualification may be under- 
line, color, line width, yellow mark, secret, personal, etc. 

The information which it to be qualified may be selected 
by passing the handheld device from the second area, to the 
10 first area and encircling the information to be qualified and 
possibly back again to the second area. 

The association may involve that first information in a 
first area is connected to second information in a second area 
into a single information. 
15 The association may be over more than two areas, multiple 

association. Thus, the handheld device is passed over three or 
more areas having discontinuous coordinates for associating 
three or more information, actions or qualifications. 

Other objects, features and advantages of the invention 
2 0 appears from the detailed description of the invention given 
below with reference to embodiments of the invention shown on 
the drawings . 

SHORT DESCRIPTION OF THE DRAWINGS 

Fig. la is a schematic diagram of a page according to the 

2 5 invention having a drawing and text made by hand. 

Fig. lb is a schematic diagram of the page of Fig. 1 
separated into three pieces. 

Fig. lc is a schematic diagram of a line drawn over two 
separated and offset pieces of the page of Fig. lb. 

3 0 Fig. 2a is a schematic diagram of a page provided with a 

send line. 

Fig. 2b is a schematic diagram of the digital representa- 
tion of the send line in Fig. 2a. 

Fig. 3 is a schematic diagram of an imaginary surface 
3 5 divided in several regions. 

Fig. 4 is a schematic diagram similar to Fig. 3a of a 
business card provided with a send line. 

Fig. 5 is a schematic diagram of a page provided with two 
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messages and two labels. 

Fig. 6 is a schematic diagram of an information handling 
system according to the invention. 

Fig. 7 is a schematic diagram of a ruler provided with 
5 boxes corresponding to different associations. 

Fig. 8 is a schematic diagram of a handheld device to be 
used according to the present invention. 
DETAILED DESCRIPTION OF EMBODIMENTS 

Fig. la discloses a paper page 111 provided with a drawing 
10 112 of a sun and a text 113, "HELLO" all drawn by hand by a 
pen on the paper. 

The paper is provided with an absolute position coding 
pattern as indicated in the small square 114 which is enlarged 
to the right. The square is provided with several dots arran- 
15 ged in a pattern as described in more detail below. Although 
only square 114 is shown with the absolute position coding 
pattern, the entire page is provided with such a pattern. The 
pattern is more or less invisible for the naked eye, since the 
dots are small, such as having a diameter of 0,08 mm, and 
2 0 arranged at a small distance from each other, such as 0,3 mm. 
The pattern is used for digitizing the movement of the pen 
over the paper. The pen is provided with an optical sensor 
which images the dots. A computer in the pen calculates x-y- 
coordinates from the image and the arrangements of the dots, 
25 as described below. Thus, a digital representation of the 

drawing 12, the sun, and the text 13, "HELLO", is provided. 

The page is divided in three parts by lines 115,116. Fig 
lb shows the page separated into three pieces. The three 
pieces may now be handled separately and moved to different 
30 locations. However, the pen and the digitalizat ion system is 
unaware of the fact that the page is separated. If the pen 
draws a new line 117,118 from one of the pieces to another of 
the pieces, which is offset from the first piece, the physical 
continuous line 117,118 on the page is represented in the 
35 digital world as a discontinuous line 117 1 , 118 1 as shown in 

Fig. lc. The discontinuity 119 represents the displacement of 
the two pieces in relation to each other. 

According to the invention, the concept of a continuous 



physical line which is represented as a digital discontinuous 
line is used for generating associations between information. 
Such a continuous physical, discontinuous digital line is 
called a hyper-line. 
5 Fig. 2 shows a simple example of this concept. A note book 

paper 221 is provided with a text 222 and a drawing 223 as 
well as address information 224, ga@cpen.com . The page is 
provided with a code pattern over the entire side, as shown by 
enlargement 22 7. 

10 Moreover, the page comprises a send button 226, which also 

is provided with an absolute position coding pattern 225, 
however, discontinuous from the code pattern of the rest of 
the page . 

The page is provided with text and drawing at a suitable 

15 time, such as during a lecture in a classroom. The address 

information is also inserted. The information is jotted down 
by a handheld device, such as a digital pen, which at the same 
time accumulates the positions which the pen assumes and form 
a digital representation of the information, which may be 

2 0 stored in the memory of the handheld device. 

When the page is completed, the user may want to send the 
page as an e-mail to the address indicated. He then draws a 
hyperline 226 joining the send button 225 with the paper 221. 
The digital pen is provided with a program which recognizes a 

2 5 hyperline and starts a program whenever a hyperline is drawn. 
The program looks at the coordinates included in the two por- 
tions of the hyperline. In this case the hyperline starts in 
paper area 221 and ends in the send box 225, but the opposite 
is also possible. The end 226b of the hyperline comprises 

30 coordinates from an area separate from the page 221. The 

program interprets these coordinates as an instruction to send 
an e-mail with the information included in an area defined by 
the rest of the line 22 6a. The area defined by the rest of the 
line, is the entire page 221. Thus, an association between the 

35 paper area 221 and the send box 225 is obtained by the hyper- 
line 226, which defines an action to be undertaken on page 
221 . 

In order to further clarify how associations may be 
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generated by hyperlines according to the invention, a further 
description of the absolute position coding pattern and the 
use thereof may be required, with reference to Fig. 3. 

The absolute position coding pattern may comprise dots 
5 arranged adjacent crossing points of a rectangular raster, 
having perpendicular raster lines with a distance of 0,3mm. 
The dots are displaced from the crossing points by a small 
distance, such as one sixth of the distance between the lines 
and in the direction of the lines. Such displacement indicates 

10 a value for each dot. Since there are four displacement direc- 
tions, each dot may represent four values, "0", "1", "2", "3". 
The raster may be virtual, in the sense that it is only used 
as a help for defining the positions of the dots. It is not 
printed on the surface together with the dots and is thus 

15 invisible. 

If an optical sensor of the pen images a surface square 
comprising 6*6 dots, these 3 6 dots may represent a numeral 
which may have 4^3 6 different values, since each dot may have 
four different values, as explained above. 

2 0 These 3 6 dots are arranged to define coordinates for a 

position of this set of dots on an imaginary surface, which 
may be very large. The coding is floating in the sense that 
each dot may take part in several squares of 6*6 dots. Such an 
imaginary surface is schematically depicted in Fig. 3. 

2 5 This large imaginary surface 3 01 may be divided in 

regions, each having assigned separate properties or uses. 
Thus, in Fig. 3 there is shown a note book region 3 03, a 
common region 304, an application region 305, a business card 
region 3 06, a diary region 3 07 and a send box region 3 08. 

3 0 Moreover, there may be a local sync region 3 02, which is 

divided in several sub regions, such as a send box region 
308', a diary region 307', a business card region 306', a note 
book region 303', a common region 304' and an application 
region 3 05'. 

35 As clearly shown in the note book region 303, the region 

is divided in several areas 309, each of a predefined size. 

In order to give an example, it is supposed that the total 
imaginary surface comprises x-y-coordinates of a binary type. 



i.e. defined by zeros and ones. The coordinates has a length 
of 36 bits for each of the x-coordinates and y-coordinates and 
codes a surface having 2^36 positions. 

The different regions may be dedicated for different 
operations or functions. In the example, the send region 308 
is dedicated to be used when generating a send command to the 
pen. For example, the send region may be defined as the region 
in which all coordinates start with x=0001.... and 

y = 0001 To be more general, each region may be defined by 

the first four bits of each of the x- and y-coordinates. In 
this manner, 256 regions may be defined. Only 6 regions are 
shown in Fig . 3 . 

As shown in the note book region 303, the region is divi- 
ded in a number of equally large areas 309. Suppose that each 
such area is slightly above 1 square meter, 12 bits are requi- 
red for defining this surface with a resolution of 0,3 mm (0,3 
mm * 2^12 = 1228,9 mm) . Thus, the last twelve bits are used 
for defining the position inside the note book page area. The 
remaining twenty bits, (36 - 4 - 12 = 20) are used for defi- 
ning the specific page area in the note book region. Thus, 
4^20 different page areas may be defined, which means 
1 099 511 627 776. 

The send box region 308 is divided in still smaller areas 
comprising 1 9 , 2mm* 1 9 , 2mm, which corresponds to 6 bits. Thus, 
the total number of send areas that may be defined is 
4^26 = 4 503 599 627 370 496. 

In Fig. 3, the areas are shown as rectangular areas having 
no overlap. The areas may have any shape. It is also possible 
to have regions and areas which overlap, but are separated by 
mathematical algorithms. 

Information about the imaginary surface and the extent of 
the regions are stored in the system, in one or several compu- 
ter systems, as described in more detail below. 

The absolute position coding pattern, which defines the 
coordinates for the above positions, may be applied at any 
product, such as paper, plastics, etc. It may also be integra- 
ted or applied to a computer screen. In the following, a 
product provided with such a pattern is called "digital 
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paper" . 

This coding pattern as well as a handheld device, such as 
a pen, is disclosed in WO 01/16691, the technical disclosure 
of which is incorporated herein by reference. 
5 The pen may comprise a program which makes it possible to 

differentiate between information which is to be stored in the 
pen, information which should be transferred to the computer 
of the user, and information which should be sent to a fax 
number or via a modem or that should be sent to a server at a 

10 predetermined IP address. Thus, a certain region comprising a 
predetermined set of coordinates may be dedicated to be trans- 
ferred to a specific recipient, for example, information 
written in the diary area 3 07 is always transferred to a 
calender program in the users personal computer or computer 

15 system. Another example is that all information written in the 
local sync region 302 is transferred to the users personal 
computer via a local sync operation and program. 

Thus, the pen and the computer system may have information 
about what the different regions at the imaginary surface are 

2 0 dedicated for. No specific component of the system needs to 
have full information about all the different regions, but 
such information may be distributed over several units. 

Fig. 4 discloses a business card 410 which is arranged to 
use the present concept of a hyperline. Only the backside of 

25 the card is shown in Fig. 4. Each human being may be assigned 
his own area in the business card region 306, comprising coor- 
dinates privately owned by him. The assignment is registered 
in several servers, for example one per country. The IP 
addresses to these servers are stored in a central server. The 

30 coordinates of this private business area is coded on the 

backside of the business card in a personal box 411. Coordina- 
tes from an send area from the send region 3 08 is coded in a 
send box 412. Finally coordinates from a common area 304 is 
coded on the rest of the business card. 

35 When using this business card for the purpose of sending 

information, the user writes a message on the backside of the 
business card in area 413, such as "Let's meet at my office at 
2 pm tomorrow" . He than indicates the e-mail address of the 



recipient in the address field 414. Finally, he draws a hyper- 
line which passes from the area 413, through area 412 to area 
411. This hyperline is interpreted by the pen program to send 
an e-mail with the message to the indicated address with the 
business card owner as sender, and possibly also a copy to the 
business card owner. It is realized that area 413 taken from 
the common area may be replaced by the private coordinates, 
whereby box 411 is superfluous. 

Moreover, it is possible to use the pen- ID, in order to 
replace the address indication. This is done by writing with 
the users pen on the recipients business card. Then the system 
is programmed to send the message to the owner of the business 
card with the owner of the pen as sender. 

It is realized that the server which holds the information 
of the private business area distribution may also comprise 
all details of the business card owner, so that a pen owner 
may obtain all such information only by making a marking at 
the personal box 411. Further variants are possible as disclo- 
sed in Swedish Patent Application No. 0003195-5, filed 
September 7, 2000, the technical disclosure of which is 
included herein by reference. 

Fig. 5 dicloses a somewhat different use of association. A 
page 510 is provided with an absolute position coding pattern 
over the entire surface. Two messages 511 "Call Eric +46 46 
123 123" and 512 "Meeting at Monday, 2 pm +46 46 123 123" are 
drawn by hand with a handheld device according to the inven- 
tion. The user has a number of labels 513,514, each having its 
own coordinates arranged thereon. The label is provided with 
an adhesive at at least a portion of the backside, and may be 
adhered to the page 510 where desired. The label may include 
printings which tells the operation which the label is asso- 
ciating to the information, such as 
TO DO and SMS . 

A hyperline 515 is drawn from the label 513 and encircling 
message 511 and returning to the lable 513. The information 
encircled by the hyperline is then associated the function of 
the label. In this case the information is transferred to the 
owners computer and inserted in a to-do-list. 



Another hyperline 516 is drawn from the second label 514 
and encircling the second message 512 and back to the label. 
The information encircled by the hyperline is then associated 
the function of the label. In this case the message is sent as 
a SMS to the telephone number indicated in the message. 

The labels may be replaced by a ruler, as shown in Fig. 7 
including several separate areas, each representing a function 
or a qualification. Thus, ruler 710 is provided with an area 
for sending a SMS 711, an area for sending an e-mail 712, an 
area for sending a fax 713, an area for storing the informa- 
tion in a specific folder called MY-folder 714. Moreover, the 
information may be qualified in different respects by a hyper- 
line, and the ruler includes qualifiers such as an area for 
underline 715, an area for italic 716, an area for color 717, 
an area for personal 718, an area for secret 719. Moreover, 
there may be associations for certain operations, such as an 
area for encryption 72 0, an area for character recognition 
721, and an area for character recognition and translation to 
a predefined language 722. Finally, there is an association 
for connection of the users v. card information to the document 
723. Further areas may be arranged such as further folders and 
further v. card information areas for different persons, such 
as collegues. 

The ruler is placed adjacent a page comprising the 
information which should be associated according to the 
present invention, and a hyperline is drawn from the ruler 
area to the information. 

The information may be encircled as mentioned in relation 
to Fig. 5. Alternatively, the entire contents of the page in 
the case of a note book page as mentioned above, may be inclu- 
ded as the message. 

Another use of the ruler is that the ruler is placed 
adjacent a paper comprising an absolute position coding 
pattern coding coordinates which are linked to information 
appearing at the page. Such an example is a printed book 
provided with such a pattern. When a paragraph is encircled in 
said book and associated with the area 714, Store in MY 
folder, the pen is arranged to start a program which looks up 



which publisher has printed the book, which is defined by the 
coordinates, and contacts the server of this publisher, and 
gives the exact coordinates of the encircled paragraph. The 
publisher has this paragraph in ASCII format in his server and 
sends this paragraph to the recipient as defined by the pen- 
ID. 

Fig. 6 discloses an embodiment of an information handling 
system in which the present invention may be used. The system 
comprises a handheld device 4 01, which in this case is a 
combined ordinary pen with a stylus that may dispense inc or 
other material leaving a permanent trace on a paper 402. The 
pen may also comprise an optical sensor for imaging a pattern 
of dots or markings provided on the paper and generating coor- 
dinates for the position of the pen on the paper. The coordi- 
nates are calculated by a microprocessor arranged in the pen 
and are stored in a memory. The pen is provided with an 
identification number 403, called pen-ID, which may be a 
manufacturing number hard coded in the memory or otherwise 
assigned to the pen. The pen- ID may not be changed. The pen is 
further described with reference to Fig. 8. 

The paper is provided with an absolute position coding 
pattern over the entire page as depicted by the enlargement 
404. The paper is provided with information 4 05, such as 
"Message 1", "Message 2", "Message 3", which may be entered by 
the pen ("Message 1" and "Message 2") or may be preprinted on 
the paper ("Message 3") . Furthermore, there is arranged boxes 
408, 409, 410 and 411, which are labeled SMS , FAX, @ and SEND. 
Finally, there are lines 412 and 413 provided with demarcation 
lines between which letters or numerals may be written. 

The pen is arranged to write on the paper, for example 
"Message 1" and "Message 2". During the writing, the pen 
senses the dots of the pattern and calculates position coordi- 
nates for the pen at regular time intervals, such as 100 times 
per second. Thus, a digital trace of the pen movement is 
formed in the pen. The pen may also, in addition to position 
information and time information, store information about the 
pressure between the pen and paper and the inclination and 
rotation of the pen in relation to the paper. The information 



is stored as a set of coordinates, which may be converted to a 
vector format . 

Information is also entered at the lines 412, 413, as 
shown "ga@cpen.com" . Such information may be e-mail addresses, 
telefax numbers, SMS telephone number etc. The information may 
also contain text to be included in the subject line of an e- 
mail . 

Then, a hyperline is drawn from the box 410 into the page 
area 402, in order to qualify the information to be sent as an 
e - ma i 1 . 

Finally, a hyperline is drawn from the send box 411 to the 
page area 402. The line portion in the send box indicates to 
the pen that a send operation should be performed, and the 
hyperline indicates that the send operation should be perfor- 
med on the information included at page 402. The previous 
hyperline indicates that the send operation should be an e- 
mail . 

The pen is in contact with the Internet via some suitable 
means not shown at Fig. 6, but further explained in connection 
with Fig. 8. A send action involves that the pen 403 sends a 
data packet comprising coordinates for identification of the 
owner of the particular coordinates for which the service 
should be performed, as indicated by line 421. In the present 
case, coordinates obtained from the box 410 is sent together 
with the pen-ID, to a server 420 at internet which is located 
at a specific IP address, which is stored in the pen. The 
coordinates from box 410 is interpreted by the server, by 
means of a database table 423, as belonging to an e-mail 
operator, such as Telia AB . Then, the server returns an IP 
address for Telia AB to the pen, as shown by line 422. 

In the next step, the pen sends a request, as indicated by 
line 432, including the pen-ID, to the server 430 for Telia 
AB, the address of which was obtained in the step outlined 
above. The server 43 0 looks in a database 431 in order to 
determine if the pen- ID has authorization and a contract for 
the requested service and sets up a receiving program, and 
finally sends an acknowledgement signal together with some 
information about which information should be transmitted, 
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such as format, size etc. Then, the pen prepares a data 
packet, with the information, in this case the e-mail address 
of the recipient, any information to be included in the 
subject line, and the very information, which may be graphical 
5 etc, which is sent to the Telia server 43 0 as indicated with 
line 434. The Telia server performs its servicing and trans- 
mits the e-mail to the recipient. 

In the situation with an e-mail, some of the above actions 
may be dispensed with, so that the message may pass directly 
,10 from the pen to the e-mail service provider. 

In another situation, the page 402 may be an advertise- 
ment, for example for company A, in a newspaper or magazine. 
In this case, the send button comprises information about the 
advertisement owner A. As indicated above, the send button is 

15 selected from the send button region which comprises 4 503 599 
627 370 496 different send buttons. That many different compa- 
nies may obtain its own send button. 

The process is the same as indicated above, except that 
the first message sent according to line 421 comprises the 

2 0 send button coordinates and the pen- ID. The server 42 0 inclu- 
des a register over send button owners in database 423 and 
returns the IP address of the advertisement owner, i.e. 
company A. Then, the pen contacts the company A server and 
sends the information as requested by that server. 

2 5 A handheld device that may be used according to the 

invention is shown in Fig. 8. The handheld device, which also 
is called a pen above, comprises a housing 811 with an opening 
812 at one end thereof. A light source 813, such as one or 
several LEDs illuminates the surface below the opening 512, 

30 normally the paper surface. An optical sensor 814 is arranged 
to sense dots or markings arranged on the paper visible 
through opening 812. There is normally an optical system such 
as lenses or mirrors. The sensor is a two-dimensional array 
and delivers an image signal to a microprocessor 816 compri- 

35 sing a memory 817. A power source such as a battery 815 is 

arranged to power the pen. A stylus 818 is arranged adjacent 
the opening 812 in order to leave a trace on the paper as an 
ordinary pen. The stylus may dispense inc which is invisible 
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by infrared light, and the LEDs and the sensor may operate at 
IR light, which means that the stylus trace do not interfere 
with the sensing of the coding pattern or dots. The pen may be 
provided with one or several buttons 819 in order to operate 
5 the pen, such as on-off switch. Moreover, the pen may be 
provided with a display screen 821. 

In order to transmit the information from the pen, it is 
provided with a short-range radio transmitter and receiver 
820, which may establish a connection with an adjacent compu- 

10 ter or mobile telephone and further to Internet. Such a short- 
range radio communication system is manufactured by Ericsson 
Radio System AB, a Swedish company, under the trademark 
BLUETOOTH. Other communication systems may be used as well, 
such as IR communication, or wire. 

15 The pen may accumulate information in the memory while not 

being connected to Internet and transmit such information when 
connection is established, for example during night time, when 
the pen is placed in a cradle. The information in the pen may 
be erased upon transmission, or may be maintained until a 
2° 2 0 delete order is given by the user. 

As is understood by the examples given above, the data 
transfer structure is very flexible and makes it possible to 
use a single server with associated database, for keeping 
track of different owners of certain services included in the 

25 system, while this server is revealed from the heavy traffic 
of the complete information transfer. 

In the above examples, a hyperline has been used for 
interconnecting or associating information with other infor- 
mation, services or operations. In the above examples, 

30 association has been used for defining which regions and areas 
that should be included in a sending operation. A hyperline is 
formed comprising coordinates from several different areas in 
one and the same line. The pen comprises a pressure sensor 
which indicates when the pen is in contact with a support and 

35 a line or a trace is formed. It is only when the pressure 

sensor is activated and the pen is in contact with the support 
that the pen senses and registers coordinate positions and at 
the same time a line is drawn on the support by the pen. This 
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physical line is continuous and extends over the surface of 
the support, such as across two overlapping pages of paper. 
The digital representation of this continuous line comprises 
one or several discontinuities when the pen passes over a 
5 borderline between non-continuous areas. Thus, a hyperline is 
a digital representation of a physically continuous line, in 
which the digital representation comprises one or several 
discontinuities . 

Thus, a single line or a continuous trace comprises coor- 

10 dinates from several different regions and/or areas. One 

example of a hyperline is the above-mentioned send line, which 
starts or ends in a send box. The send box indicates that a 
send action should be undertaken by the pen. The send program 
of the pen may be arranged to include coordinates from each 

15 region or area of the hyperline, in the information packet 
generated. The coordinates from the send box indicates that 
the pen is to initiate a send function to a specific pre- 
programmed IP address in which a server is positioned, and 
which comprises a database over the different subscribers of 

2 0 the system each defined by the coordinates of the send box. 

The coordinates from the remaining regions or areas are used 
by an application or a program in the server or computer which 
is to receive the information, and indicates to this server or 
computer which program is to be started, such as an e-mail 

2 5 program. 

A hyperline may be used in other occasions than as a send 
line. Thus, a hyper line may be used for qualifying certain 
information . 

One example is that a user's messages jotted down at a 
30 notebook page is to be sent by an e-mail to a recipient. The 
user has a business card from the receiver with the personal 
coordinates of the business card owner at the backside. 
Instead of writing an e-mail address, the user positions the 
backside of the business card on the note book page and draws 
35 a hyperline from the business card to the note book page so 

that the hyperline registers coordinates from both areas. When 
this information is analyzed by an e-mail program, which is to 
send the information, the e-mail program looks for a hyperline 
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which may give information about an e-mail address and finds 
this hyperline. The program sends a question or request to an 
internet server about which person the business card coordi- 
nates belong to and receives the business card owners e-mail 
5 address in return. 

Several note book pages may be linked by a hyperline, for 
example by drawing a hyperline across both pages or several 
pages arranged beside each other. 

The user may have predefined a number of e-mail addresses 

10 in certain personal areas in the users private area, so that a 
hyperline from such a predefined area results in that the 
program searches in a specific database after the preprogram- 
med e-mail address. Also the pen-ID may be used for personal 
information, such as the fact that all e-mail communication 

15 should be sent as a copy to the pen-owners e-mail address as a 
confirmation . 

The user may also have preprogrammed other operations 
which by means of a hyperline may be associated with informa- 
tion in different manners. A user may have stored all his 

2 0 previous notes at a private personal computer and wants to 

find a certain page. He has retained a small piece of said 
page and puts it beside the function area which he has 
predefined as a search function. A hyperline between the 
function area and said piece establishes an association 
25 between the function area and the note book page. The search 
program can now easily locate the page by means of the 
coordinates of said piece and may display it on the screen. 
This search function may be extended to find information 
stored at a server somewhere on Internet, i.e. information 

3 0 floating in the "cyber space". 

Further examples of the search function is the following: 
In an article the author gives a reference list at the end of 
the article, where each reference is coded with an absolute 
position coding pattern according to the present invention. By 
3 5 drawing a hyperline between the readers business card and the 
coding pattern of the reference, an association is generated 
which may result in that the reference is sent to the business 
card owner via e-mail. By arranging an absolute position 
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coding pattern adjacent each author, such a hyperline may- 
involve that predetermined information about the author is 
sent to the business card owner. 

An association may be generated between more than two 
5 areas, such as described above. Thus, a send request may 

comprise a send box from a send region, one or several note 
book pages from a note book region, personal information from 
a business card region and payment information from a payment 
region, etc. 

10 Association may also be used for qualifying the infor- 

mation to be sent. If a hyperline passes a secret area, this 
means that the pen, when analyzing the hyperline, interprets 
that the information to be sent should be encrypted by a 
predefined algorithm. Other operations may also be qualified 

15 by hyperlines, such as the fact that the information is to be 
character recognized and possibly subsequently translated into 
another language. The association may be interpreted by the 
pen or by the program which handles the information trans- 
mitted by the pen. 

20 If an association is interpreted by the pen, it may be 

used for activating different operation modes of the pen, such 
as enabling encryption as mentioned above. 

The present invention has been described with an absolute 
position coding pattern comprising dots arranged according to 

25 a rectangular raster. Any type of absolute position coding 

pattern may be used with the invention as soon as such pattern 
may be arranged adjacent each other so that a discontinuity is 
defined by a single line. However, the pattern also needs to 
have a portion with continuous coordinates so that the discon- 

3 0 tinuity may be discovered as something which is out of order. 
In this way, the handheld device can easily recognize the 
hyperline and immediately undertake action, if that was the 
intention. Moreover, a hyperline may easily be discovered in a 
message or a piece of information which normally have conti- 

3 5 nuous coordinates. 

Hereinabove, the invention has been described with 
reference to several embodiments of the invention. A skilled 
person realizes that the features included in the different 
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embodiments may be combined in other combinations than the 
above-mentioned combinations and such combinations are inten- 
ded to be within the scope of the invention. The invention is 
only limited by the appended claims. 



